Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

os: optimise cgo clearenv #70672

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

os: optimise cgo clearenv #70672

wants to merge 1 commit into from
+87 −3

Conversation

cyphar
Copy link

@cyphar cyphar commented Dec 4, 2024 •
edited
Loading

For programs with very large environments, calling unsetenv(3) for each
environment variable can be fairly expensive because of CGo overhead,
but clearenv(3) is much faster. The only thing we have to track is
whether GODEBUG is being unset by the operation, which can be done very
quickly without resorting to doing unsetenv(3) for every variable.

This change makes os.Clearenv() ~30% faster when run an environment with
1000 environment variable set (and gets better with more environment
variables):

cpu: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
       │   old.txt   │               new.txt               │
       │   sec/op    │   sec/op     vs base                │
Env-16   213.6µ ± 1%   185.1µ ± 3%  -13.32% (p=0.000 n=10)

(The time taken to set up 1000 variables in this benchmark is 117.2µs,
so while the naive timing is 13% the speedup for os.Clearenv() itself is
from 96.4µs to 67.9µs which is ~30%.)

Note: Unfortunately, not all POSIX platforms support clearenv(3).
glibc/Linux has had it for a long time, but Darwin doesn't have it and
FreeBSD only got it in FreeBSD 14. So for now, we will only implement
this for Linux, but it should be trivial to add this for other platforms
that have clearenv(3).

@gopherbot
Copy link
Contributor

This PR (HEAD: a606b0f) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/633515.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from Gopher Robot:

Patch Set 1:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@cyphar cyphar mentioned this pull request Dec 4, 2024
Open
@gopherbot
Copy link
Contributor

Message from Ian Lance Taylor:

Patch Set 1:

(3 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@cyphar cyphar changed the title os/clearenv: optimise cgo clearenv os: optimise cgo clearenv Dec 5, 2024
@cyphar
os: optimise cgo clearenv
71446e9 
For programs with very large environments, calling unsetenv(3) for each
environment variable can be fairly expensive because of CGo overhead,
but clearenv(3) is much faster. The only thing we have to track is
whether GODEBUG is being unset by the operation, which can be done very
quickly without resorting to doing unsetenv(3) for every variable.

This change makes os.Clearenv() ~30% faster when run an environment with
1000 environment variable set (and gets better with more environment
variables):

    cpu: AMD Ryzen 7 7840U w/ Radeon  780M Graphics
           │   old.txt   │               new.txt               │
           │   sec/op    │   sec/op     vs base                │
    Env-16   213.6µ ± 1%   185.1µ ± 3%  -13.32% (p=0.000 n=10)

(The time taken to set up 1000 variables in this benchmark is 117.2µs,
so while the naive timing is 13% the speedup for os.Clearenv() itself
the speedup is from 96.4µs to 67.9µs which is ~30%.)

Change-Id: I589794845854e1721e6fc5badc671474c2bbd037
@cyphar cyphar force-pushed the os-clearenv-optimisation branch from a606b0f to 71446e9 Compare December 5, 2024 02:29
@gopherbot
Copy link
Contributor

This PR (HEAD: 71446e9) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/633515.

Important tips:

  • Don't comment on this PR. All discussion takes place in Gerrit.
  • You need a Gmail or other Google account to log in to Gerrit.
  • To change your code in response to feedback:
    • Push a new commit to the branch used by your GitHub PR.
    • A new "patch set" will then appear in Gerrit.
    • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
    • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
    • Multiple commits in the PR will be squashed by GerritBot.
  • The title and description of the GitHub PR are used to construct the final commit message.
    • Edit these as needed via the GitHub web interface (not via Gerrit or git).
    • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
  • See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

@gopherbot
Copy link
Contributor

Message from Aleksa Sarai:

Patch Set 1:

(4 comments)

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Ian Lance Taylor:

Patch Set 3: Commit-Queue+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 3:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2024-12-05T04:30:15Z","revision":"19cbf5e28344a2b4c3819f31ce0d98cfad575db6"}

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Ian Lance Taylor:

Patch Set 3: -Commit-Queue

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 3:

This CL has passed the run

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Go LUCI:

Patch Set 3: LUCI-TryBot-Result+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Ian Lance Taylor:

Patch Set 3:

(1 comment)

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

@gopherbot
Copy link
Contributor

Message from Kirill Kolyshkin:

Patch Set 3: Code-Review+1

Please don’t reply on this GitHub thread. Visit golang.org/cl/633515.
After addressing review feedback, remember to publish your drafts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@cyphar @gopherbot